Improving multi-robot teleoperation by inferring operator distraction

A high-quality human-robot interface is essential for the success of search and rescue operations in urban environments that are too challenging for fully-autonomous operation. Teleoperating multiple robots greatly increases the complexity of the human’s cognitive task, since the operator’s concentration is divided among multiple robots. Thus, simply adding more robots to the system does not necessarily expand the effective coverage region nor increase the rate at which the operator can search. We present CoOperator, an agent-based human-robot interface that infers operator distraction and identifies any robots that are not currently being effectively managed. A CoOperator agent assumes control of such a robot and moves it along a search path that complements the operator’s explicit teleoperation. The CoOperator agent seamlessly cedes control to the user whenever direct commands are given and resumes directing the robot if the operator’s attention shifts. We demonstrate that our agents significantly improve multi-robot teleoperation through user studies on four urban search and rescue scenarios with a team of three simulated Pioneer 3DX robots.